3 research outputs found
Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation
Understanding and manipulating deformable objects (e.g., ropes and fabrics)
is an essential yet challenging task with broad applications. Difficulties come
from complex states and dynamics, diverse configurations and high-dimensional
action space of deformable objects. Besides, the manipulation tasks usually
require multiple steps to accomplish, and greedy policies may easily lead to
local optimal states. Existing studies usually tackle this problem using
reinforcement learning or imitating expert demonstrations, with limitations in
modeling complex states or requiring hand-crafted expert policies. In this
paper, we study deformable object manipulation using dense visual affordance,
with generalization towards diverse states, and propose a novel kind of
foresightful dense affordance, which avoids local optima by estimating states'
values for long-term manipulation. We propose a framework for learning this
representation, with novel designs such as multi-stage stable learning and
efficient self-supervised data collection without experts. Experiments
demonstrate the superiority of our proposed foresightful dense affordance.
Project page: https://hyperplane-lab.github.io/DeformableAffordanc
Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects
Articulated object manipulation is a fundamental yet challenging task in
robotics. Due to significant geometric and semantic variations across object
categories, previous manipulation models struggle to generalize to novel
categories. Few-shot learning is a promising solution for alleviating this
issue by allowing robots to perform a few interactions with unseen objects.
However, extant approaches often necessitate costly and inefficient test-time
interactions with each unseen instance. Recognizing this limitation, we observe
that despite their distinct shapes, different categories often share similar
local geometries essential for manipulation, such as pullable handles and
graspable edges - a factor typically underutilized in previous few-shot
learning works. To harness this commonality, we introduce 'Where2Explore', an
affordance learning framework that effectively explores novel categories with
minimal interactions on a limited number of instances. Our framework explicitly
estimates the geometric similarity across different categories, identifying
local areas that differ from shapes in the training categories for efficient
exploration while concurrently transferring affordance knowledge to similar
parts of the objects. Extensive experiments in simulated and real-world
environments demonstrate our framework's capacity for efficient few-shot
exploration and generalization
Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions
Perceiving and manipulating 3D articulated objects in diverse environments is
essential for home-assistant robots. Recent studies have shown that point-level
affordance provides actionable priors for downstream manipulation tasks.
However, existing works primarily focus on single-object scenarios with
homogeneous agents, overlooking the realistic constraints imposed by the
environment and the agent's morphology, e.g., occlusions and physical
limitations. In this paper, we propose an environment-aware affordance
framework that incorporates both object-level actionable priors and environment
constraints. Unlike object-centric affordance approaches, learning
environment-aware affordance faces the challenge of combinatorial explosion due
to the complexity of various occlusions, characterized by their quantities,
geometries, positions and poses. To address this and enhance data efficiency,
we introduce a novel contrastive affordance learning framework capable of
training on scenes containing a single occluder and generalizing to scenes with
complex occluder combinations. Experiments demonstrate the effectiveness of our
proposed approach in learning affordance considering environment constraints.
Project page at https://chengkaiacademycity.github.io/EnvAwareAfford/Comment: In 37th Conference on Neural Information Processing Systems (NeurIPS
2023). Website at https://chengkaiacademycity.github.io/EnvAwareAfford